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FINAL  REPORT  ON  QNR  CONTRACT  N00014-75-C-0586 

ENTITLED  "STATISTICAL  ENGINEERING" 

This  is  the  FINAL  REPORT  on  ONR  Contract  N00014-75-C-0586  entitled 
"Statistical  Engineering"  with  Dr.  Robert  Bechhofer  as  Principal  Investi¬ 
gator.  The  contract  was  initiated  on  June  1,  1975  and  was  renewed  five 
times  through  March  31,  1982.  ANNUAL  REPORTS  were  submitted  to  cover  the 
following  periods:  FIRST  REPORT  (6/1/75-8/31/75),  SECOND  REPORT  (9/1/75- 
8/31/76),  THIRD  REPORT  (9/1/76-8/31/77).  In  addition,  a  SPECIAL  REPORT 
(requested  by  Dr.  E.J.  Wegman,  Director  of  the  Statistics  and  Probability 
Program)  summarizing  the  research  accomplishments  achieved  on  the  contract 
was  submitted  on  8/14/78.  Copies  of  these  reports  are  attached.  This 
FINAL  REPORT  describes  research  accomplishments  subsequent  to  the  SPECIAL 
REPORT..  Thus  collectively  these  reports  summarize  progress  to  date. 

A\ 

This 'FINAL  REPORT  i.s  divided  into  three  sections  as  follows: 

1.  Background  for  the  research  undertaken  on  ONR  Contract 
N00014-75-C-0586. 
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Recent  developments 
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1.  3ackground  for  the  research  undertaken  on  ONR  Contract  N00014-75-C-0586 

The  major  thrust  of  research  on  this  contract  has  been  on  the  development 
of  statistical  procedures  for  "selecting  or  ordering"  populations  or  for  making 
"multiple  comparisons"  among  populations.  Some  background  concerning  the  early 
history  of  research  in  this  area,  and  of  the  Principal  Investigator's  role  in 
advocating  and  initiating  the  development  of  such  procedures  (particularly  ones 
employing  the  so-called  "indifference-zone"  approach  to  ranking  and  selection 
problems)  is  given  in  Part  I  (pages  2-5)  of  the  SPECIAL  REPORT.  The  following 
paragraphs  which  sketch  the  setting  for  much  of  the  research  on  this  contract 
were  provided  by  the  Principal  Investigator  for  inclusion  in  the  report  titled 
"Program  Summary  for  FY  SI"  (edited  by  Edward  J.  Wegman,  Program  Director)  on 
the  Statistics  and  Probability  Program  (ONR  Code  436)  dated  June  1981;  see  page 
21  of  that  report. 

"Engineers  or  scientists  engaged  in  procurement,  research  and  development,  or 
testing  are  often  faced  by  the  problem  of  selecting  the  "best"  of  several  compet¬ 
ing  categories,  e.g.,  the  best  product  or  type  or  system,  or  brand,  or  treatment. 
Thus,  a  procuring  activity  might  be  sampling  the  product  of  several  competing 
contractors  prior  to  the  award  of  a  production  contract  (in  order  to  select  the 
one  with  the  smallest  fraction  defective)  or  an  ordnance  engineer  involved  in 
research  and  development  might  be  conducting  firing  programs  to  compare  the  bal¬ 
listic  performance  of  different  types  of  projectiles  (in  which  case  his  objective 
might  be  to  select  that  type  which,  on  the  average,  has  the  deepest  penetration 
or  the  longest  range),  or  a  systems  engineering  might  be  comparing  the  performance 
of  several  competing  systems  to  select  the  one  with  the  highest  reliability,  or 
a  ohemica!  engineer  might  be  conducting  experiments  to  compare  the  performance  of 
different  brands  of  measuring  equipment  (in  which  case  his  objective  might  be  to 


select  that  brand  which  has  the  highest  precision,  i.e.,  the  greatest  reproducibility). 
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For  each  of  these  examples,  the  objective  of  the  experiment  or  study  might 
have  been  more  comprehensive  than  simply  to  identify  the  "best"  category; 
for  example,  it  might  have  been  to  rank  the  categories  from  "best"  to  "worst" 
according  to  some  criterion  of  "goodness."  The  selection  is  to  be  done  in 
each  case  on  the  basis  of  data  obtained  in  a  controlled  test  or  experiment. 

To  accomplish  this  objective  the  experimenter  requires  a  statistical  decision 
procedure  which  tells  how  to  lay  out  the  experiment  (i.e.,  what  experimental 
design  to  use),  how  many  observations  to  take,  how  to  take  these  observations, 
and  based  on  these  observations  which  category  to  select  as  "best."  The 
decision  procedure  is  to  have  the  property  that  a)  the  probability  of  a  correct 
selection  is  controlled  at  some  specified  level,  and  b)  the  number  of  observations 
required  to  achieve  this  objective  is  minimized . 

In  other  situations  the  experimenter  might  be  interested  in  comparing  simul¬ 
taneously  two  or  more  test  categories  (or  treatments)  with  a  control  category  or 
a  standard.  Here  too  the  experimenter  may  wish  to  control  at  some  prespecified 
level  the  confidence  coefficient  associated  with  the  joint  interval  estimates  of 
the  differences  between  the  treatments  and  the  control,  and  to  do  so  with  a 
minimum  total  number  of  observations . 

Such  procedures,  called  "statistical  selection  procedures"  and  "multiple 
comparisons  procedures,"  respectively,  have  been  studied  extensively  at  Cornell 
(and  elsewhere)  over  a  period  of  years,  and  research  on  them  is  continuing." 

2 .  Recent  developments 

a .  General  deve lopments  in  this  research  area 

Interest  in  the  "selection  and  ordering"  approaches  and  in  the  "multiple 
comparisons"  approaches  has  continued  at  a  high  level.  Since  our  SPECIAL  REPORT 
in  August  1978,  two  new  books  have  appeared  which  deal  specifically  with  such 
methodologies.  These  are  Multiple  Decision  Procedures  ( Theory  and  Methodology 


published  by  John  Wiley,  1979  and  Nonparametric  Sequential  Selection 
Procedures  by  H.  Biiringer,  H.  Martin  and  K.-H.  Schriever  published  by 
Birkhauser,  1980.  In  addition.  The  Complete  Categorized  Guide  to  Statistical 
Selection  and  Ranking  Procedures  by  E.J.  Dudewicz  and  J.O.  Kco  is  in  press 
being  published  by  American  Sciences  Press,  1982.  Also,  a  substantial 
Bibliography  on  Selection  Procedures  listing  some  638  entries  had  been 
compiled  earlier  by  Professor  Gunnar  Kulldorff,  Institute  of  Mathematics  and 
Statistics,  University  of  Umea,  Sweden,  although  this  volume  dated  January  26,  1977 
is  not  generally  available.  A  comprehensive  review  of  the  book.  Selecting  and 
Ordering  Populations:  A  New  Statistical  Methodology  by  J.  Gibbons,  I.  Olkin 
and  M.  Sobel  published  by  John  Wiley,  1977,  was  undertaken  by  the  Principal 
Investigator;  it  appears  in  the  Journal  of  the  American  Statistical  Association, 

75  (1980),  751-756. 

A  short  course  on  "Selecting  and  Ordering  I opulations"  sponsored  by  the 
American  Statistical  Association  was  presented  at  the  Annual  Meeting  of  the 
Association,  August  11-12,  1979;  the  participants  in  the  course  were  R.  Bechhofer, 
J.  Gibbons,  S.S.  Gupta  and  I.  Olkin.  Videotapes  of  that  course  will  be  shown 
(as  a  short  course)  at  the  upcoming  meeting  of  the  Association  in  Cincinnati 
on  August  14-15,  1982.  R.  Bechhofer  and  S.S.  Gupta  offered  a  5-day  seminar  on 
"Selecting  and  Ranking  Procedures"  in  Berlin,  West  Germany,  on  November  17-21, 

1980  sponsored  by  The  George  Washington  University  and  AMK  3erlin.  The  Third 
Conference  on  Statistical  Decision  Theory  and  Related  Topics ,  organized  by 
S.S.  Gupta  and  J.  Berger,  was  held  at  Purdue  University,  June  1-5,  1981;  several 
sessions  were  devoted  to  ranking  and  selection  procedures,  the  Principal 
Investigator  participating  in  one  of  them.  On  May  3,  1932  the  Principal 
Investigator  presented  an  invited  paper  in  a  plenary  session  at  a  meeting  of 


-5- 


the  Society  for  Clinical  Trials  held  in  Pittsburgh,  Pennsylvania  in  which 
he  described  how  a  Bernoulli  statistical  selection  procedure,  recently 
developed  by  him  and  Dr.  Radhika  Kulkarni  (see  below),  might  be  used 
effectively  in  medical  trials.  Thus  the  potential  of  ranking  and  selection 
procedures  continues  to  capture  the  imagination  of  both  theoreticians  and 
practitioners  with  more  and  more  interest  being  shown  in  the  subject.  As  a 
consequence,  the  literature  in  the  field  continues  to  grow  at  a  great  rate, 
b .  Some  specific  research  accomplishments  achieved  on  ONR  Contract 
N00014-75-C-0536 

In  this  section  we  will  focus  our  attention  on  two  particular  research 
accomplishments  recently  achieved  on  this  contract.  These  are  i)  The  develop¬ 
ment  of  a  new  class  of  incomplete  block  designs,  which  we  term  BTIB  (balanced 
treatment  incomplete  block)  designs,  for  comparing  simultaneously  several  test 
treatments  with  a  control  treatment,  and  ii)  The  development  of  closed  adaptive 
sequential  procedures  for  selecting  the  best  of  k  >.  2  Bernoulli  populations. 

We  shall  comment  on  each  of  these  below, 
i)  BTIB  designs 

In  a  series  of  technical  reports  (TR's  414,  425,  436,  440,  441,  and  453 
listed  in  Section  3a)  of  the  present  report)  the  Principal  Investigator  and 
Tamhane  proposed  a  new  class  of  incomplete  block  designs  (BTI3  designs)  for 
comparing  several  test  treatments  with  a  control  treatment;  these  designs 
generalize  the  notion  of  a  balanced  incomplete  block  (BIB)  design.  TR  414 
described  the  properties  that  such  designs  must  posses,  including  a  necessary 
and  sufficient  condition  concerning  the  structure  of  these  designs.  It  was 
shown  that  for  the  multiple  comparisons  with  a  control  problem,  attention 
could  be  limited  to  unions  of  such  BTI3  designs,  and,  in  particular,  to  a 
subset  of  such  designs  which  was  termed  the  "minimal  complete  class  of  generator 
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designs."  This  class  depends  on  p  =  (the  number  of  test  treatments  being 
compared  with  the  control  treatment)  and  k  =  (the  common  block  size).  The 
determination  of  this  minimal  complete  class  is  a  difficult  combinatorial 
problem  which  was  considered  for  various  (p  ,k ) -combinations  in  the  remaining 
TR's  in  the  series;  once  the  minimal  complete  class  is  known  for  any  particular 
(p,k)-combination,  it  is  possible  to  find  an  optimal  BTIB  design  for  any  given 
b  =  (number  of  blocks).  The  general  theory  underlying,  these  BTIB  designs  was 
published  in  Technometrics ,  1981,  after  which  a  paper  giving  the  optimal  designs 
for  p  =  2(1)6,  k  =  2  and  p  =  3,  k  =  3  was  accepted  for  publication  in 
Sankhya'.  The  problem  of  constructing  the  minimal  complete  class  of  generator 
designs  for  p  >  3,  k  =  3  and  p  ^  4,  k  =  4  aroused  the  interest  of  several 
combinatorial  experts.  Professor  William  I.  Notz  of  Purdue  University  solved 
this  problem  for  p  =  3(1)10,  k  =  3  while  T.E.  Ture,  a  Ph.D.  student  at 
Berkeley,  solved  it  for  p  =  4(1)10,  k  =  4  and  for  larger  k-values  with 
?  >.  k,  as  well.  Notz  demonstrated  additional  optimal  properties  (in  the  sense 
of  the  optimal  design  criteria  of  the  late  Jack  Kiefer)  of  these  designs. 
Professor  C.-F.  Wu  of  Wisconsin  is  now  also  studying  these  designs.  Thus, 
starting  with  a  practical  problem  for  which  no  good  answer  had  heretofore  been 
provided,  the  Principal  Investigator  and  Tamhane  have  opened  up  a  new  area  of 
theoretical  research  with  interesting  experimental  applications.  The  Principal 
Investigator  and  Tamhane  have  prepared  tables  of  optimal  BTIB  designs  which  are 
under  consideration  for  publication  in  the  Selected  Tables  in  Mathematical 
Statistics  series. 

ii)  Closed  adaptive  sequential  procedures  for  selecting  the  best  of  k  ^  2 
Bernoulli  populations 

In  two  technical  reports  (TR  510  and  TR  532  iisted  in  Section  3a'  of  the 
present  report)  the  Principal  Investigator  and  Kulkarni  proposed  closed  adaptive 
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seauential  procedures  for  selecting  the  best  of  k  >.  2  Bernoulli  populations. 

In  TR  510  the  authors  consider  two  general  goals:  Goal  I  (selecting  the  s 
best  of  k  without  regard  to  order)  and  Goal  II  (selecting  the  s  best  of  k 
with  regard  to  order);  here  (1  <_  s  k-1,  k  >_  2).  In  TR  532  they  focus  on 
the  particular  special  case  s  =  1,  k  ^  2,  in  which  case  the  two  goals  coincide. 
The  point  of  departure  for  these  procedures  is  the  Sobel-Huyett  [1957]  single- 
stage  procedure  which  takes  exactly  n  observations  from  each  of  the  k 
populations.  The  Bechhofer-Kulkarni  (B-K)  procedures  (which  were  described  in 
a  Ph.D.  dissertation  by  Ms.  Kulkarni  written  under  the  guidance  of  the 
Principal  Investigator)  take  no  more  than  n  observations  from  each  of  the 
k  populations,  and  achieve  exactly  the  same  probability  of  a  correct  selection 
as  does  the  S-H  procedure,  uniformly  in  the  k  unknown  Bernoulli  "success" 
probabilities.  Moreover,  for  k  =  2  the  B-K  procedure  is  optimal  in  the  sense 
that  if  minimizes  the  expected  total  number  of  observations  taken  from  both 
populations  at  termination  within  a  broad  class  of  completing  procedures;  for 
k  =  2  it  possesses  additional  optimality  properties  as  well.  It  now  appears 
that  in  certain  regions  of  the  k-dimensional  cube  (0  <_  p^  <_  1,  1  <  i  <  k) 
which  represents  the  parameter  space  for  this  problem,  the  procedure  is  also 
optimal  for  k  >  2;  these  regions  are  very  easy  to  describe. 

The  B-K  procedure,  which  is  stationary,  requires  no  special  tables  for 
implementation,  is  very  easy  to  carry  out  in  practice,  and  always  requires  less 
observations  (usually  many  less)  on-the-average  compared  to  the  single-stage 
procedure  which  achieves  the  same  probability  of  a  correct  selection.  Moreover, 
the  3-K  procedure  tends  to  sample  less  frequently  from  the  Bernoulli  populations 
with  small  p-values,  thus  making  it  ethically  desirable  for  use  in  clinical 
trials  where  small  p-values  are  associated  with  treatments  with  low  probability 


of  "success."  A  comprehensive  study  of  the  performance  characteristics  of 
the  3-K  procedure  was  carried  out  in  TR  532;  quantitative  assessments  of  its 
"goodness"  were  obtained.  This  procedure  has  provoked  considerable  interest 
among  both  practitioners  and  theoreticians.  Research  on  its  properties,  and 
on  variations  of  the  procedure,  is  continuing. 

Some  of  the  results  in  TR  510  were  presented  at  the  Third  Conference  on 
Statistical  Decision  Theory  and  Related  Topics  held  at  Purdue  University  in 
June  1981;  the  entire  paper  will  appear  shortly  in  the  Proceedings  of  the 
Conference . 


3 .  Research  supported  in  whole  or  in  part  by  ONR  Contract  N00014-75-C-0586 
a .  Technical  reports 

Bechhofer,  R.E.  and  Santner,  T.J.:  "A  note  on  the  lower  bound  for 
the  P{JS}  of  Gupta's  subset  selection  procedures,"  TR  401 
(January  1979 ,  revised  December  1979). 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for 
comparing  treatments  with  a  control:  general  theory,"  TR  414 
(March  1979). 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for 
comparing  treatments  with  a  control  (II):  optimal  designs  for 
p  =  2(1)6,  k  =  2  and  p  =  3,  k  =  3,"  TR  425  (May  1979). 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for 
comparing  treatments  with  a  control  (III):  optimal  designs 
for  p  =  4,  k  =  3  and  p  =  5,  k  =  3,"  TR  436  (October  1979). 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for 
comparing  treatments  with  a  control  (IV):  optimal  designs 
for  p  =  4,  k  =  4,"  TR  440  (January  1980). 

3echhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for 
comparing  treatments  with  a  control  (V):  optimal  designs  for 
p  =  6,  k  =  3,"  TR  441  (June  1980). 

Bechhofer,  R.E.,  Tamhane,  A.C.  and  Mykytyn,  S.W.:  "Incomplete  block 

designs  for  comparing  treatments  with  a  control  (VI):  conjectured 
minimal  class  of  generator  designs  for  p  =  5,  k  =  4,  and 
d  =  6,  k  =  4,"  TR  453  (April  1980). 
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McCulloch,  C.E.:  "Conditions  under  which  E{Nj_}  =  00  for  Tong's 
adaptive  solution  to  ranking  and  selection  problems," 

TR  480  (September  1980). 

Bechhofer,  R.E.  and  Tamhane ,  A.C.:  "Tables  of  optimal  allocations 
of  observations  for  comparing  treatments  with  a  control," 

TR  489  (January  1981). 

Bechhofer,  R.E.  and  Dunnett,  C.W.:  "Multiple  comparisons  for 
orthogonal  contrasts,"  TR  495  (March  1981). 

Bechhofer,  R.E.  and  Kulkarni,  R.V.:  "Closed  adaptive  sequential 
procedures  for  selecting  the  best  of  k  >.  2  Bernoulli 
populations,"  TR  510  (July  1981). 

Faltin,  F.W.  and  McCulloch,  C.E.:  "On  the  small-sample  properties 

of  the  Olkin-Sobel-Tong  estimator  of  the  probability  of  correct 
selection,"  Florida  State  University  Statistics  Report  No.  M581 
(July  1981).  Based  on  research  done  while  the  authors  were  at 
Cornell  University. 

Bechhofer,  R.E.  and  Kulkarni,  R.V.:  "On  the  performance  characteristics 
of  a  closed  adaptive  sequential  procedure  for  selecting  the  best 
Bernoulli  population,  TR  532  (April  1982). 


b .  Papers  published  or  accepted  for  publication 

Barton,  R.R.  and  Turnbull,  B.W.:  "A  survey  of  covariance  models  for 
censored  life  data  with  an  application  to  recidivism  analysis," 
Communications  in  Statistics--Theory  and  Methods,  A8(8),  1979, 

723-750. 

Bechhofer,  R.E. :  Invited  review  of  Selecting  and  Ordering  Populations — 

A_  New  Statistical  Methodology  by  J.  Gibbons,  I.  Olkin  and  M.  Sobel 
(John  Wiley  1977),  Journal  of  the  American  Statistical  Association, 

75  (1980),  751-756. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  comparing 
treatments  with  a  control:  general  theory,"  Technometrics ,  23  (1981), 
45-57.  Corrigendum:  Technometrics ,  24  ( 1982 ) ,  171 . 

Bechhofer,  R.E.  and  Kulkarni,  R.V.:  "Closed  adaptive  sequential  procedures 
for  selecting  the  best  of  k  ^  2  Bernoulli  populations,"  Statistical 
Decision  Theory  and  Related  Topics-III  (Eds.  S.S.  Gupta  and  J.  Berger), 
New  York,  Academic  Press,  1982. 

3echhofer,  R.E.  and  Dunnett,  C.W.:  "Multiple  comparisons  for  orthogonal 
contrasts."  To  appear  in  Technometrics ,  24  (1982). 
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Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Design  of  experiments  for 
comparing  treatments  with  a  control:  tables  of  optimal 
allocations  of  observations."  Accepted  for  publication 
in  Technometrics ,  subject  to  minor  revision. 

Sechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs 
for  comparing  treatments  with  a  control  (II):  optimal 
designs  for  p  =  2(1)6,  k  =  2  and  p  =  3,  k  =  3." 

Accepted  for  publication  in  Sankhya~,  subject  to  minor 
revision. 

Faltin,  F.W.:  "Performance  of  the  Sobel-Tong  estimator  of  the 
probability  of  correct  selection  achieved  by  Bechhofer's 
single-stage  procedure  for  the  normal  means  problem." 

Abstract  80t-59,  Bulletin  of  the  Institute  of  Mathematical 
Statistics ,  9  (1980),  180. 

Faltin,  F.W.:  "A  quantile  unbiased  estimator  of  the  probability 
of  correct  selection  achieved  by  Bechhofer's  single-stage 
procedure  for  the  two  population  normal  means  problem," 

Abstract  80t-60,  Bulletin  of  the  Institute  of  Mathematical 
Statistics ,  9  (1980),  180-181. 

Hooper,  J.H.  and  Santner,  T.J.:  "Design  of  experiments  for  selection 
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SPECIAL  REPORT  ON  ONR  CONTRACT  N00014- 75-C-0586 

This  is  a  special  report  on  ONR  contract  N00014-75-C-0586  (and  its  prede¬ 
cessor  contract)  with  Dr.  Robert  Bechhofer  as  Principal  Investigator.  The 
report  deals  with  the  last  five  years  of  these  contracts.  It  consists  of  three 
main  parts:  Part  I  gives  some  of  the  history  of  the  general  area  of  statistical 
"ranking  and  selection"  procedures,  and  of  the  Principal  Investigator's  association 
with  research  in  that  area  which  is  the  one  that  has  received  major  attention  on 
this  contract;  a  precis  of  the  most  significant  research  accomplishments  obtained 
on  this  contract  in  the  last  five  years  is  also  included  in  this  part.  Part  II 
lists  published  papers  and  unpublished  technical  reports  supported  by  the  contract 
during  the  last  five  years;  these  are  grouped  into  three  categories.  Part  III 
discusses  the  directions  of  research  to  be  undertaken  by  the  Principal  Investigator 
over  the  next  3  to  5  years,  and  lists  specific  problems  which  the  Principal 
Investigator  plans  to  study. 


PART  I  OF  REPORT 


Background 

The  Principal  Investigator  has  been  engaged  in  research  in  the  general 
area  of  statistical  multiple-decision  procedures  with  particular  emphasis  on 
"ranking  and  selection"  procedures  and  multiple  comparisons  procedures  for 
many  years.  In  an  early  paper  [ B 1 ]  he  proposed  the  "ranking  and  selection" 
approach  as  an  alternative  to  the  classical  "test  of  homogeneity"  approach 
(using,  for  example,  the  F-test  associated  with  the  analysis  of  variance) 
since  he  felt,  based  on  his  experience  —  in  applied  statistics,  that  the 
latter  formulation  of  the  problem  does  not  answer  the  questions  that  truly 
are  of  interest  to  an  experimenter.  When  the  performances  of  several  competing 
categories  of  test  items  are  being  compared,  what  the  experimenter  usually 
wishes  to  ascertain  is  which  category  or  categories  are  best,  and  whether  or 
not  the  best  ones  are  good  enough  to  select,  or  he  may  be  interested  in 
estimating  the  differences  between  the  performances  of  items  in  each  category. 
Such  questions  can  be  answered  more  meaningfully  if  the  statistical  problems 
are  posed  at  the  outset  as  ones  involving  "ranking  or  selection"  or  "multiple 
comparisons,"  and  the  experiment  designed  accordingly.  Properly  planned 
experiments  with  well  defined  objectives  such  as  these  often  result  in  con¬ 
siderable  savings  in  sample  size  and  associated  costs. 

Most  of  the  research  undertaken  by  the  Principal  Investigator  under  ONR 
sporsorship  has  proceeded  with  this  orientation  as  a  central  philosophy.  He 

If 

Dr.  Bechhofer  served  for  four  years  (during  World  War  II)  as  a  statistician 
at  the  Aberdeen  Proving  Ground,  Maryland;  he  was  responsible  for  the  design, 
analysis,  and  interpretation  of  _ests  of  items  of  ordnance.  He  has  partic¬ 
ipated  regularly  in  the  annual  Design  of  Experiments  Conference  sponsored  by 
the  Army  Research  Office-Durham.  Following  his  stay  at  Aberdeen,  Dr.  Bechhofer 
was  employed  as  a  statistician  at  the  Carbide  and  Carbon  Chemical  plant  in 
Oak  Ridge,  Tennessee. 


along  with  several  co-workers ,  research  students,  and  other  independent  research 

workers  throughout  the  world  have  been  developing  various  integrated  approaches 

to  the  meaningful  formulations  and  solution  of  ranking  and  selection  problems. 

The  two  most  widely  adopted  formulations  of  these  problems  are  the  so-called 

"indifference- zone"  approach  (first  proposed  by  the  Principal  Investigator  in 
2/ 

[Bl]—  ),  and  the  "subset"  approach  (first  proposed  by  Dr.  Shanti  S.  Gupta  in 
[A7 ] ,  [A8] ) .  A  "restricted  subset"  approach,  which  bridges  the  indifference- 
zone  and  the  subset  approaches  was  proposed  by  Dr.  Thomas  J.  Santner  (14].  Recently, 
Dr.  Ajit  Tamhane  and  the  Principal  Investigator  proposed  a  procedure  [B26] ,  [B30] 
which  provides  a  different  blending  of  the  two  basic  approaches.  Several  other 
formulations  have  also  been  set  forth.  Considerable  interest  has  been  shown  in 
these  formulations,  and  at  the  present  time  more  than  400  research  papers  have 
been  written  on  this  general  subject  and  published  in  the  technical  journals. 

A  brief  overview  of  this  general  area  of  statistical  research  (with  particular 
emphasis  on  the  indifference-zone  approach),  and  of  some  of  the  Principal 
Investigator's  contributions  to  it,  are  contained  in  a  expository  paper  [B24] . 

The  research  monograph  [B13]  by  Bechhofer,  Kiefer,  and  Sobel  gives  a  fairly 
complete  set  of  references  up  to  1968.  An  up  to  date  categorized  bibliography 
on  ranking  and  selection  procedures  [A3]  by  Dudewicz  and  Koo  lists  more  than  400 
references . 

The  book  [A2]  by  Dudewicz  is  one  of  the  first  elementary  texts  to  introduce 
ranking  and  selection  procedures  as  one  of  the  fundamental  forms  of  statistical 
analysis  available  to  the  applied  statistician.  Kleijnen  in  his  book  [A10]  on 

j? 

—  References  in  this  report  prefixed  by  B  refer  to  the  papers  of  Bechhofer 
listed  on  pp. 6-8  while  references  prefixed  by  A  refer  to  additional  papers 
by  other  investigators  listed  alphabetically  on  pp.  9-10. 
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statistical  techniques  in  simulation  devotes  more  than  200  pages  to  showing  the 
substantial  role  that  selection  and  ranking  procedures  and  multiple  comparisons 
procedures  can  play  in  the  design  and  analysis  of  simulation  experiments.  A 
major  step  forward  with  respect  to  making  the  ranking  and  selection  methodology 
available  to  the  practitioner  in  a  useable  form  took  place  with  the  recent  publi¬ 
cation  of  the  book  [A6]  by  Gibbons,  Olkin,  and  Sobel;  as  noted  on  its  dustjacket, 
"This  book  is  the  first  of  its  kind  to  provide  applied  statisticians  and  other 
users  of  statistical  methods  with  techniques  for  selecting  and  ordering  (or  ranking) 
populations."  Many  of  the  procedures  developed  by  the  Principal  Investigator  and 
his  co-workers  are  described  in  detail  in  this  text.  Finally,  a  major  monograph 
[A9]  by  Gupta  and  Panchapakesan  at  the  research  level  is  in  final  draft  stage, 
and  will  shortly  be  published;  this  book  surveys  and  integrates  the  significant 
results  in  the  field,  and  contains  a  comprehensive  bibliography. 

As  these  new  techniques  take  their  place  in  the  textbooks  alongside  the 
traditional  methods  of  design  and  analysis  of  experiments,  there  has  been 
increasing  demand  on  the  part  of  practitioners  to  learn  more  about  this  approach 
and  its  applicability  in  various  areas  of  experimentation.  In  December  1977  the 
Principal  Investigator  was  invited  to  give  a  three-hour  tutorial  on  ranking  and 
selection  procedures  at  the  Thirty-Third  Annual  Princeton  Conference  on  Applied 
Statistics;  this  session  was  attended  by  more  than  350  individuals  most  of  whom 
expressed  considerable  interest  in  the  subject;  many  mentioned  that  the  procedures 
were  very  relevant  to  work  that  they  were  doing. 

Most  recently,  on  June  8-9,  1978,  the  Principal  Investigator  conducted  a 
two-day  tutorial  seminar  on  "Statistical  Selection  and  Ranking  Procedures"  at 
the  Ballistic  Research  Laboratory,  Aberdeen  Proving  Ground,  Maryland.  This 
tutorial  which  was  sponsored  by  the  Army  Research  Office-Durham  with  the  coop¬ 
eration  of  Drs.  Malcolm  S.  Taylor  and  James  Richard  Moore  of  the  BRL  attracted 
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approximately  fifty  attendees;  about  two-thirds  were  from  Aberdeen  or  Edgewood 
Arsenal  with  nine  additional  installations  being  represented.  The  response  of 
the  attendees  to  the  subject  matter  presented  was  extremely  positive;  most  of 
the  group  indicated  a  strong  desire  to  learn  more  about  the  topics  discussed 
so  that  they  could  employ  them  in  their  own  work. 
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SPECIAL  RESEARCH  ACCOMPLISHMENTS 


The  earliest  research  on  statistical  ranking  and  selection  procedures  saw 
particular  emphasis  being  placed  on  single-stage  procedures  involving  single¬ 
factor  experiments.  (see,  e.g.,  [Bl],  [B3],  [B6],  [AS],  [A18]).  Although 
some  significant  progress  was  made  in  the  development  of  sequential  procedures 
for  single-factor  experiments  involving  normal  means  ([A12]),  it  was  not  until 
the  publication  of  [B13]  that  an  integrated  general  theory  was  presented  for 
the  solution  of  a  broad  class  of  such  problems.  Single-stage  procedures  for 
multi-factor  e:  eriments  concerned  with  normal  means  were  considered  briefly 
in  [Bl]  and  [Al],  while  single-stage  procedures  for  multi-factor  experiments 
concerned  with  normal  variances  were  considered  in  [B14],  The  problem  of 
optimal  allocation  of  observations  when  comparing  several  treatments  with  a 
control  (posed  by  Dunnett  [A4])  was  first  solved  in  [B16].  Most  of  the  special 
research  accomplishments  of  recent  years  represent  substantial  improvements  on 
or  generalizations  of  procedures  mentioned  above. 
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DISCUSSIONS  OF  SPECIFIC  RESEARCH  ACCOMPLISHMENTS 


Single- factor  experiments 

A)  Two- stage  procedures  for  selecting  the  largest  normal  mean  when  the 
common  variance  is  known 

In  [B26] ,  [B30]  the  authors  propose  a  two-stage  procedure  with 
screening  to  select  the  normal  population  with  the  largest  population  mean 
when  the  populations  have  a  common  known  variance.  The  procedure  guarantees 
the  same  probability  requirement  using  the  indifference- zone  approach  as  does 
the  single-stage  procedure  of  Bechhofer  [B 1 ] .  It  has  the  highly  desirable 
property  that  the  expected  total  number  of  observations  required  by  the  procedure 
is  always  less  than  the  total  number  of  observations  required  by  the  corresponding 
single-stage  procedure  [Bl],  regardless  of  the  configuration  of  the  population 
means .  The  saving  in  expected  total  number  of  observations  can  be  substantial, 
particularly  when  the  configuration  of  the  population  means  is  favorable  to  the 
experimenter.  The  saving  is  accomplished  by  screening  out  "non-contending" 
populations  in  the  first  stage,  and  concentrating  samr.r,.ng  only  on  "contending" 
populations  in  the  second  stage.  The  second  paper  [B30]  contains  new  results 
which  make  possible  the  more  efficient  implementation  of  the  two-stage  procedure. 
Tables  for  this  purpose  are  given,  and  the  improvements  achieved  (which  are 
significant)  are  assessed. 

B)  Single-stage  and  two-stage  procedures  for  selecting  the  largest  normal 
mean  when  the  common  variance  is  known  or  unknown,  respectively,  and  comparisons 
are  made  with  a  fixed  known  standard 

In  [B29]  the  authors  propose  a  single-stage  and  a  two-stage  (k+l)-decision 
procedure  for  the  case  of  common  known  and  common  unknown  variance,  respectively. 
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for  the  problem  of  comparing  k  normal  means  with  a  specified  (absolute) 
standard.  The  procedures  guarantee  that  i)  with  probability  at  least  P* 
(specified),  no  population  is  selected  when  the  largest  population  mean  is 
sufficiently  less  than  the  standard,  and  ii)  with  probability  at  least  P* 
the  population  with  the  largest  population  mean  is  selected  when  that  mean 
is  sufficiently  greater  than  its  closest  competitor  and  the  standard.  Tables 
to  implement  the  procedures  are  provided.  Such  procedures  are  applicable  in 
the  following  type  of  situation:  If  the  competing  categories  (normal  populations) 
are  methods  of  heat  treating  steel,  then  the  best  method  may  not  be  deemed 
satisfactory  unless  the  expected  tensile  strength  resulting  from  that  method  of 
treatment  is  at  least  some  specified  minimum  value. 

C)  Adaptive  sequential  procedures  for  selecting  the  largest  normal  mean 
when  the  common  variance  is  known 

In  the  monograph  [B13],  sequential  vector-at-a-time  sampling  procedures 
are  given  for  selecting  the  largest  of  k  .>  2  normal  means  when  the  common 
variance  is  known,  and  the  so-called  "indifference- zone"  formulation  of  the 
ranking  problem  is  adopted.  However,  in  certain  types  of  experimental  situations 
vector-at-a-time  sampling  procedures  are  not  appropriate,  e.g.,  in  biomedical 
clinical  trials  it  may  be  desirable  to  concentrate  sampling  on  contending  popu¬ 
lations  which  are  indicated  as  being  superior  (i.e.,  having  large  population 
means)  and  sample  less  frequently  or  eliminate  from  sampling  non-contending  popu¬ 
lations  which  are  indicated  as  being  inferior  (i.e.,  having  small  population 
means).  A  two-population  problem  in  which  adaptive  sample  was  employed  had 
earlier  been  studied  in  [A13] .  In  the  present  paper  [A19]  the  authors  consider 
the  corresponding  k-population  (k  ^  5)  problem,  and  employ  various  adaptive  (non 
vector-at-a-time)  sampling  rules.  They  show  that  unlike  the  case  k=2,  when  k^5 


substantial  savings  in  expected  total  sample  size  can  be  achieved  when  adaptive 
sampling  is  employed  for  the  so-called  identification  problem  (as  opposed  to 
the  ranking  problem);  the  alternative  criterion  of  minimizing  expected  number  of 
observations  on  the  inferior  populations  is  also  studied  for  the  identification 
problem.  Both  theoretical  and  simulation  results  are  presented.  However,  if 
these  same  adaptive  procedures  are  used  for  the  corresponding  "indifference- zone" 
formulation  of  the  ranking  problem,  the  authors  obtain  the  somewhat  surprizing 
result  that  the  slippage  configuration  is  no  longer  necessarily  "least-favorable, 
and  thus  the  usual  indifference- zone  probability  requirement  for  the  ranking  prob 
lem  may  not  be  guaranteed.  Hence,  adaptive  sampling  procedures  for  the  ranking 
problem  involving  k  >.  3  populations  remain  to  be  developed. 

r>)  A  single-stage  procedure  for  selecting  the  normal  mean  which  is  closest 
to  a  specified  value  when  the  common  variance  is  known 

In  [B27]  the  authors  propose  a  single-stage  procedure  for  selecting  the 
production  process  with  the  highest  fraction  of  conforming  product;  if  the 
underlying  variates  from  each  of  the  k  ^  2  populations  are  normally  distributed 
with  a  common  known  variance,  and  if  two-sided  specification  limits  are  adopted 
for  the  individual  items,  then  the  problem  reduces  to  the  one  described  in  the 
title  of  this  section.  In  [B27]  both  parametric  and  non-parametric  procedures 
are  described,  and  an  abbreviated  set  of  tables  to  implement  the  parametric 
procedure  is  given.  The  paper  [B27]  is  expository.  A  paper  giving  the  under¬ 
lying  theory  and  an  expanded  set  of  tables  is  being  written  up  with  Dr.  Turnbull. 


E)  Optimal  allocation  of  observations  when  comparing  treatments  with  a 
control  in  incomplete  blocks  (normal  means,  common  known  variance) 

In  [B32]  the  authors  are  preparing  two  papers  which  generalize  the 
results  of  [B16]  to  the  case  in  which  (as  in  many  biomedical  experiments) 
observations  can  only  be  taken  in  incomplete  blocks.  All  of  the  necessary 
theory  has  now  been  developed,  and  the  results  are  being  written  up  for  publication. 
The  first  paper  gives  exact  optimal  allocations  for  p  =  2(1)6  "test"  treat¬ 
ments  which  are  to  be  compared  to  a  "control"  treatment  in  b  blocks  of  size 
k  =  2,3,4  (k  <  p+1)  where  b  (which  depends  on  p  and  k)  is  roughly  less  than 
100;  the  second  paper  gives  approximate  continuous  (in  the  sense  of  Kiefer) 
optimal  allocations  for  the  same  (p,k)- combinations  where  b  is  arbitrary  (but 
usually  fairly  large).  The  computations  for  these  papers  are  almost  completed. 


F)  Exact  confidence  intervals  fo^-Qi^-p-^  in  2x2  contingency  tables 


In  [A15]  the  authors  have  used  an/exact  confidence  interval  estimate 
of  the  "odds-ratio"  ii  =  p^l-p^/p^l-pj  in  a  2x2  contingency  table  (where 
the  p^  (i  =  1,2)  are  Bernoulli  "success"  probabilities)  to  obtain  an  exact 
confidence  interval  estimate  of  the  difference  A  =  p^-p-,  and  of  the  relative 
risk  d  *  p,/p,.  The  question  of  how  Vo  form  exact  (small- sample)  confidence 

A.  *m  " 

intervals  on  these  latter  quantities  has  been  an  open  problem  for  many  years. 
Tables  to  implement  the  procedure  are  provided. 


16 


Multi- factor  experiments 

I.  Studies  in  which  the  experiment  is  designed  for  the  purpose  of  ranking 
or  selecting  "interactions"  (normal  distributions  with  a  common  known  variance) 

A.  Single-stage  procedures  for  selecting  the  largest  interaction  in  a 
2- factor  experiment 

i)  In  [B25]  the  authors  propose  a  single-stage  procedure  for  selecting 
the  largest  positive  interaction  in  a  2- factor  experiment  involving  qualitative 
variables.  The  procedure  would  be  applicable  in  the  following  type  of  situation: 
"Suppose  that  a  medical  research  worker  wished  to  plan  an  experiment  to  study 
the  effect  of  several  (c)  different  methods  of  treatment  on  a  physiological 
response  of  male  and  female  subjects.  It  is  assumed  known  that  the  effect  of 
the  treatment  on  the  mean  response  is  different  for  men  than  for  women,  and  also 
that  it  varies  from  treatment  to  treatment.  It  is  suspected  that  there  may  be  a 
large  interaction  between  sex  and  method  of  treatment,  and  it  is  desired  to 
identify  the  sex-treatment  combination  for  which  this  interaction  is  largest  in 
the  hope  that  such  information  might  provide  some  clue  as  to  the  mechanism  under¬ 
lying  the  effectiveness  of  the  methods  of  treatment."  The  statistical  problem  is 
to  design  the  experiment  on  such  a  scale  that  this  largest  interaction  can,  if  it 
is  sufficiently  large  to  be  of  practical  importance  to  the  experimenter,  be  detected 
with  preassigned  probability.  (More  generally,  in  a  2-factor  experiment  there 
might  be  r  >_  2  levels  of  one  qualitative  factor  and  c  ^  2  levels  of  a  second 
qualitative  factor.)  The  present  paper  considers  2-factor  experiments  and 
concentrates  on  the  2*c  case  and  the  3x3  case.  The  probability  of  a  correct 
selection  is  evaluated  for  the  given  procedure.  The  main  result  of  the  paper  is 
to  give  the  least- favorable  configuration  for  the  cases  under  study. 


• — wrnr 
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ii)  In  [A16]  the  author  extends  the  work  in  [B25] .  The  paper  analyzes 
the  least- favorable  configuration  based  on  the  log-concavity  of  the  probability 
of  a  correct  selection  regarded  as  a  function  of  the  population  interactions  and 
on  the  characteristics  of  the  preference  zone.  The  3x4  case  is  examined  in 
detail  and  explicit  results  are  given  for  that  case.  Nonlinear  programming 
algorithms  for  finding  the  extreme  points  associated  with  the  least- favorable 
configuration  are  given  for  the  general  rxc  case. 

II.  Studies  in  which  the  experiment  is  designed  for  the  purpose  of  ranking 
or  selecting  "main  effects"  (normal  distributions  with  a  common  unknown  variance). 
Note:  Here  a  different  procedure  must  be  used  according  as  one  assumes  that 
interaction  j^s  not  or  present. 

A.  No  interaction  between  factors 

In  [B28]  single-stage  and  two-stage  selection  procedures  are  given  for 
two- factor  experiments  in  which  the  common  variance  is  unknown.  The  objective 
is  to  rank  on  both  factors  (Factors  A  and  B,  say)  simultaneously.  The  paper  [B28] 
is  expository;  a  small  set  of  abbreviated  tables  to  implement  the  procedures  is 
given.  A  three- part  paper  giving  the  underlying  theory  is  being  prepared  as  a 
joint  effort  with  Dr.  Charles  W.  Dunnett  of  McMaster  University,  Canada;  the 
papers  will  contain  a  comprehensive  set  of  definitive  tables  necessary  to  implement 
the  procedures.  The  computation  of  these  tables  is  nearing  completion. 
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PART  III  OF  REPORT 
Directions  of  Future  Research 

It  is  anticipated  that  the  investigations  to  be  undertaken  will  proceed 
along  lines  similar  to  those  followed  during  the  earlier  periods.  A  large 
number  of  areas  of  research  remain  to  be  explored.  Special  emphasis  will  be 
placed  on  solving  problems  which  have  practical  relevance--many  of  them  having 
been  brought  to  the  attention  of  the  Principal  Investigator  during  his  periods 
as  a  practicing  statistician  in  various  aspects  of  experimental  design.  Every 
effort  will  be  made  to  formulate  the  problems  in  a  way  that  will  be  meaningful 
and  useful  to  the  practitioner,  and  to  devise  procedures  which  will  be  easy  to 
implement. 

The  main  objective  of  this  contract  is  to  make  contributions  to  the  meaningful 
formulation  and  solution  of  statistical  problems  which  can  be  usefully  viewed  as 
ranking  and  selection,  or  related  problems.  These  can  be  grouped  into  two  broad 
classes:  a)  Problems  arising  from  single- factor  experiments,  and  b)  Problems 
arising  from  multifactor  experiments.  Some  problems  associated  with  single¬ 
factor  experiments  have  an  exact  analogue  in  multifactor  experiments;  however, 
other  problems  are  unique  to  multifactor  experiments  because  the  appropriate 
statistical  model  to  use  for  multifactor  experiments  is  often  not  obvious,  and 
each  choice  leads  to  a  different  definition  of  "interaction."  We  discuss  some 
of  these  problems  below. 

A)  Single- factor  experiments 

i)  Problems  concerned  with  normal  means 

a)  The  development  of  methods  for  dealing  with  unbalanced  experiments  when 
ranking  means  of  normal  populations  with  a  common  known  variance  using  a 
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single-stage  procedure.  In  the  basic  paper  [Bl],  and  in  later  papers,  consider¬ 
ation  was  given  as  to  how  large  the  common  number  of  observations  per  population 
should  be  chosen  in  order  to  guarantee  a  prescribed  probability  requirement. 
However,  practical  considerations  may  often  limit  the  number  of  observations  that 
can  be  taken  from  particular  populations  and/or  observations  may  be  lost  or  in 
error  (in  which  case  one  would  discard  them) ;  the  net  result  is  that  the  final 
experiment  is  unbalanced  in  that  different  numbers  of  observations  are  taken  from 
the  various  populations.  If  this  happens  then  the  original  prescribed  probability 
requirement  is  no  longer  guaranteed,  and  it  is  necessary  to  assess  the  magnitude 
of  the  decrease  in  the  probability  of  a  correct  selection.  The  problem  is 
particularly  complicated  if  observations  are  taken  in  (say)  randomized  blocks 
(which  here  can  be  regarded  as  a  second  factor),  and  some  blocks  are  incomplete. 

It  is  planned  to  develop  easily- implementable  procedures  for  coping  with  such 
situations.  Conservative  procedures  presently  exist  for  some  of  these  problems, 
but  these  procedures  are  often  very  inefficient.  For  some  cases  it  may  be  pos¬ 
sible  to  devise  optimal  procedures. 

b)  The  development  of  improved  procedures  for  estimating  the  achieved 
probability  of  a  correct  selection  (PCS)  in  retrospective  studies.  If  an 
experiment  has  been  conducted  and  the  sample  taken  it  will  often  be  of  interest 
to  estimate  the  probability  of  making  a  correct  selection  using  these  data  and 
a  given  selection  procedure  (e.g.,  the  single-stage  procedure  of  [Bl]).  This 
important  problem  was  posed  by  Olkin,  Sobel,  and  Tong  in  [All],  and  their  proposed 
solution  is  incorporated  into  the  text  [A6]  of  Gibbons,  Olkin,  and  Sobel. 
Unfortunately,  their  procedure  tends  to  overestimate  the  achieved  PCS  (often  by 

a  considerable  amount),  particularly  when  the  number  (k)  of  populations  is  large, 
the  common  sample  size  is  small,  and  the  k  population  means  are  all  almost  equal. 
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(In  this  situation  the  sample  means  behave  almost  as  order  statistics  from  a 
common  population.)  It  is  hoped  to  develop  an  estimator  with  improved  small- 
sample  properties. 

c)  The  development  of  a  procedure  for  optimally  allocating  observations  to 
populations  when  ranking  means  of  normal  populations  with  known  unequal  variances 
using  a  single-stage  procedure.  In  [Bl]  an  optimal  solution  to  this  problem  is 
given  for  the  case  of  two  populations,  and  a  general  solution  (which  is  known  to 
be  non-optimal)  is  given  for  the  case  of  three  or  more  populations.  It  is  hoped 
to  devise  a  procedure  which  will  be  optimal  for  this  latter  case,  or  minimally 
one  which  will  improve  substantially  on  the  present  solution. 

d)  The  development  of  a  procedure  for  optimally  allocating  observations 
among  p  >  2  test  treatments  and  a  control  treatment  when  it  is  desired  to  compare 
simultaneously  the  p  test  treatments  with  the  control  treatment.  This  problem 
which  was  originally  posed  by  Dunnett  [A4]  was  solved  for  the  case  of  completely 
randomized  designs  in  [B16]  and  [B20] .  We  propose  to  consider  this  problem  for 
the  situation  in  which  the  experimenter  is  restricted  to  taking  observations  in 
incomplete  blocks  (which  often  is  the  case  in  biomedical  experiments).  The 
Principal  Investigator,  working  with  A.C.  Tamhane,  has  solved  the  one-sided 
problem  (the  analogue  of  [ Bl 6] ) ;  it  is  planned  to  complete  the  writeup  of  this 
problem  and  then  to  work  on  the  two-sided  problem  (the  analogue  of  [B20]). 
Successful  completion  of  this  study  will  result  in  substantial  decreases  in  the 
sample  sizes  traditionally  taken  in  experimental  situations  requiring  multiple- 
comparisons  with  a  control. 

e)  The  development  of  sequential  selection  procedures  which  concentrate 
sampling  on  contending  populations,  and  which  eliminate  or  sample  less  frequently 
from  non-contending  populations.  This  problem  is  of  particular  practical  interest 
(but  of  great  analytical  difficulty)  when  three  or  more  populations  are  involved. 


Recently  Turnbull,  Kaspi  and  Smith  [A19]  obtained  some  very  interesting  results 
for  a  variant  of  the  identification  problem  but  their  solution  does  not  carry 
over  to  the  corresponding  ranking  problem.  In  an  as  yet  unpublished  paper  Richard 
L.  Smith,  a  graduate  student  at  Cornell,  was  able  to  improve  significantly  on  the 
Paulson-Fabian  [A12]-[A5]  sequential  procedure  for  the  ranking  problem. 

f)  The  development  of  single-stage  selection  procedures  for  means  of  censored 
normal  populations  when  the  underlying  variances  are  known  and  equal.  Such  pro¬ 
cedures  would  be  very  useful  in  certain  types  of  reliability  studies.  A  sequential 
procedure  for  identifying  the  normal  population  with  the  largest  population  mean, 
all  populations  having  a  common  known  variance  and  a  common  known  upper  point  of 
censoring  is  described  in  [B13,  p.  102].  However,  no  ranking  procedures  have  thus 
far  been  devised  for  this  problem. 

g)  The  development  of  lower  bounds  on  the  probability  of  a  correct  selection 
when  single-stage  selection  procedures  are  used.  It  is  recognized  that  statistical 
procedures  which  are  devised  under  one  set  of  statistical  assumptions  may  behave 
poorly  if  these  assumptions  are  not  satisfied  in  the  situation  in  which  the  pro¬ 
cedures  are  actually  applied.  The  purpose  of  this  study  will  be  to  assess  just 
how  much  the  probability  of  a  correct  selection  associated  with  the  standard  pro¬ 
cedure  of  [Bl]  is  affected  when  the  statistical  assumptions  are  violated  in  certain 
ways.  This  study  will  throw  light  on  the  "robustness"  of  such  procedures,  and  thus 
on  their  general  practicability.  Research  on  this  problem  is  underway  [B23] ,  and 
promising  preliminary  results  have  been  obtained  for  certain  special  cases.  What 
is  required  in  one  of  the  simplist  special  cases  is  a  new  generalization  (different 
from  that  of  Olkin-Marshal 1)  of  the  well-known  one-sided  Chebyshev  inequality. 

ii)  Problems  concerned  with  Bernoulli  probabilities  of  "success" 

a)  The  development  of  a  two- stage  procedure  with  elimination  for  selecting 


the  Bernoulli  population  with  the  largest  single-trial  "success"  probability. 


The  single-stage  procedure  for  this  problem  using  the  indifference- cone  approach  was 
devised  by  Sobel  and  Huyett  [A18] .  It  is  proposed  to  use  the  same  "measure  of 
distance"  as  was  employed  by  Sobel  and  Huyett  [A18] ,  and  to  devise  two-stage 
procedures  which  are  analogues  for  Bernoulli  populations  of  these  used  for 
normal  populations  by  Tamhane  and  Bechhofer  [B26],  [B30] .  It  is  hoped,  using 
this  approach,  to  improve  uniformly  on  the  procedure  of  Sobel  and  Huyett, 
b)  The  development  of  a  procedure  for  obtaining  exact  joint  confidence 
interval  estimates  of  the  "odds-ratio"  =  pQ  (I-p^/p^CI-Pq)  (1  <  i  <  c) 

in  2xc  contingency  tables.  If  this  problem  can  be  solved  for  c  >3,  then 
the  result  can  be  used  to  determine  exact  joint  confidence  interval  estimates  of 
the  differences  A.  =  p  -p.  and  relative  risks  p.  =  p./p„  (1  <  i  <  c).  A 
method  for  using  the  solution  for  ^  to  generate  a  solution  for  A^  and 
has  recently  been  proposed  by  Santner  and  Snell  [A1S]  for  c  =  2.  It  is  now 
planned  to  see  whether  their  methods  can  be  extended  to  solve  the  general  multiple 
comparisons  with  a  control  problem  for  the  Bernoulli  distribution. 

iii)  Problems  concerned  with  exponential  populations 
The  development  of  procedures  for  selecting  the  exponential  population  with 
the  largest  mean  when  data  are  subject  to  Type  I  or  Type  II  censoring.  Early 
work  on  the  selection  problem  for  exponential  distributions  was  done  by  Sobel 
[A17] ;  a  special  form  of  Type  II  censoring  was  considered  in  that  paper.  (See 
also  [B13,  pp.  268-269].)  Results  for  this  problem  would  be  useful  in  reliability 
studies  and  in  biomedical  research. 


i)  Problems  concerned  with  normal  means 

a)  In  a  two-factor  experiment  with  both  factors  qualitative  one  can  dis¬ 
tinguish  the  following  situations:  I)  It  is  known  that  there  is  no  interaction 
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(or  negligible  interaction)  between  the  two  factors,  or  II)  It  is  known  that 
there  is  large  interaction  between  the  two  factors,  or  III)  The  magnitude  of 
the  interaction  between  the  two  factors  is  completely  unknown  (i.e. ,  it  may 
be  negligible,  intermediate  or  large). 

If  I)  is  the  situation,  and  it  is  desired  to  rank  on  both  factors 
simultaneously  (i.e.,  to  find  the  best  level  of  Factor  A  and  simultaneously 
to  find  the  best  level  of  Factor  B) ,  then  it  is  known  (Bawa  [A1 ] )  that  the  most 
efficient  design  (in  the  sense  of  minimizing  the  total  number  of  observations 
required,  subject  to  guaranteeing  a  probability  requirement)  for  the  case 
of  common  known  variance  is  a  factorial  experiment.  The  Principal  Investigator 
is  currently  studying  this  problem  with  Dr.  Charles  W.  Dunnett  of  McMaster 
University  for  the  case  in  which  the  populations  have  a  common  unknown  variance. 
Most  of  the  preliminary  theoretical  work  has  been  completed  (see  [B28]  for  a 
sketch  of  the  procedures),  and  tables  to  implement  the  procedures  are  being 
prepared. 

If  II)  is  the  situation,  then  it  is  not  meaningful  .to  rank  on  both  factors 
simultaneously.  Either  the  experimenter  can  seek  to  find  the  combination  (level 
of  Factor  A  and  level  of  Factor  B)  which  is  best  (in  which  case  procedures 
already  exist  (e.g.,  [Bl]  and  [B2])  0£  the  experimenter  can  seek  to  find  the 
best  level  of  Factor  A,  simultaneously,  for  each  level  of  Factor  B.  For  the 
case  in  which  the  populations  have  a  common  unknown  variance,  the  second  problem 
requires  new  tables  to  implement  the  procedures;  these  are  being  prepared  with 
Dr.  Dunnett. 

If  III  is  the  situation,  then  the  Principal  Investigator  and  Dr.  Dunnett 
are  proposing  an  entirely  new  procedure  involving  a  preliminary  test  on  the 
interactions  to  assess  their  magnitude:  if  the  interactions  are  indicated  as 
being  negligible,  then  the  experimenter  will  proceed  as  in  I)  while  if  they 
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are  indicated  as  being  large,  then  the  experimenter  will  proceed  as  in  II). 

This  complicated  composite  procedure  is  presently  being  studied  with  Dr.  Dunnett 

b)  For  situation  I  (of  a),  above)  the  sequential  procedure  of  [B13]  has 
been  generalized  so  that  the  experimenter  can  rank  simultaneously  on  both 
factors.  Preliminary  studies  indicate  that  analogous  savings  can  be  achieved 
using  the  sequential  procedure  as  were  noted  by  Bawa  [Al]  for  the  single-stage 
procedure. 

c)  The  problem  of  jointly  estimating  efficiently  all  of  the  interactions 
in  a  multifactor  experiment  is  also  being  studied.  The  analytical  work  for  the 
two-factor  problem  has  been  completed.  Special-purpose  tables  to  implement  the 
procedure  remain  to  be  computed. 

ii)  Problems  concerned  with  Bernoulli  probabilities  of  "success." 

The  development  of  models  and  procedures  for  designing  and  analyzing  multi¬ 
factor  experiments  involving  Bernoulli  random  variables.  Such  methodology  would 
be  useful  in  devising  experiments  to  study  the  dependence  of  (say)  the  fraction 
defective  (of  manufactured  items)  on  the  various  "levels"  of  the  qualitative 
factors  (e.g.,  different  manufacturers  or  different  methods  of  manufacture) 
which  affect  the  performance  of  these  items,  or  to  study  the  dependence  of  (say) 
the  survival  probability  of  cancer  patients  on  the  "levels"  of  the  qualitative 
factors,  e.g.,  stage  of  the  disease  (early,  intermediate,  advanced)  vs. 
different  methods  of  treatment,  which  affect  this  probability.  Sequential 
procedures  such  as  the  one  described  in  [B13,  p.  270]  are  currently  being 
investigated  for  setups  in  which  the  p„  are  assumed  to  obey  a  loglinear  model 

iii)  Problems  concerned  with  multinomial  probabilities  of  "occurence." 

The  development  of  models  and  procedures  for  designing  and  analyzing  multi¬ 
factor  experiments  involving  multinomial  random  variables.  Such  methodology 
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would  be  useful  in  designing  certain  classes  of  experiments  which  lead  to  data 
in  the  form  of  contingency  tables.  Single-stage  and  sequential  procedures  for 
single- factor  experiments  are  given,  e.g.  in  [B6]  and  [B13,  pp.  121-123], 
respectively.  It  is  proposed  to  generalize  these  and  other  procedures  to 
multi- factor  experiments,  and  to  study  their  properties. 
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when  comparing  several  treatments  with  a  control  (IV):  Incomplete  block 
designs . " 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (k+1 )-decision  single-stage 
procedure  for  selecting  the  production  process  with  the  highest  fraction 
of  conforming  product." 

Santner,  T.J.  :  "Selecting  the  treatment  combination  having  the  largest 
interaction:  arbitrary  r*c  case." 

Snell,  M.K.  and  Santner,  T.J.  :  "Exact  small  sample  confidence  intervals 
for  P^~?0  and  p^/p^  i-n  2*2  contingency  tables." 

Professional  activities  of  Dr.  Bechhofer  during  the  reporting  period 

a)  Director  of  School  of  Operations  Research  6  Industrial  Engineering, 
Cornell  University. 

b)  Member  of  the  Advisory  Board  of  the  Section  on  Physical  and  Engineering 
Sciences,  American  Statistical  Association. 

c)  Member  of  the  Committee  on  Statistics  in  the  Physical  Sciences  of  the 
Bernoulli  Society  for  Mathematical  Statistics  and  Probability. 

d)  Ad  Hoc  Member  of  Computer  and  Biomathematical  Sciences  Study  Section 
of  the  National  Institutes  of  Health,  Washington,  D.C.,  November  10-12, 

1976. 

e)  Refereed  research  proposals  for  Army  Research  Office,  Durham,  and 
for  the  National  Science  Foundation. 

f)  Panelist  at  Twenty-second  Conference  on  the  Design  of  Experiments  in 
Army  Research,  Development,  and  Testing  heid  at  Harry  Diamond  Laboratories, 
Adelpni,  Md.,  October  20-22,  1976. 
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g)  Gave  an  invited  paper  "Sampling  plans  for  testing  combination  drugs" 
before  the  joint  meeting  of  the  American  Statistical  Association  and  the 
Bicmetric  Society.  ENAR  and  WNAR  held  in  Chicago,  Illinois,  August  15-13, 


SECOND  ANNUAL  REPORT  ON  CONTRACT  N00014-75-C-0586 


"STATISTICAL  ENGINEERING" 


by 


Robert  E.  Bechhofer 
Principal  Investigator 


School  of  Operations  Research  and  Industrial  Engineering 
College  of  Engineering 
Cornell  University 
Ithaca,  New  York  14853 


SECOND  ANNUAL  REPORT  ON  CONTRACT  N00014-75-C-0586 


This  is  the  second  annual  report  on  Contract  N00014-75-C-0586  which  is 
titled  "Statistical  Engineering."  The  contract  was  initiated  on  June  1,  1975 
and  continued  through  May  31,  1976  with  a  budget  of  $20,000;  the  first  renewal 
of  the  contract  covers  the  period  June  1,  1976  through  December  31,  1976  with 
a  budget  of  $15,000.  The  present  report  covers  the  period  September  1,  1975 
through  August  31,  1976.  During  summer  1976  the  contract  provided  partial 
support  to  the  Principal  Investigator,  Dr.  Robert  Bechhofer,  to  two 
Co- Investigators ,  Dr.  Thomas  Santner  and  Dr.  Bruce  Turnbull  (both  being  Assist¬ 
ant  Professors  in  the  School  of  Operations  Research  and  Industrial  Engineering 
at  Cornell),  and  to  several  graduate  research  assistants.  In  addition 
Miss  Mary  K.  Vickers  and  Messrs.  Ray  Jakobovits  and  Yehuda  Vardi  were  supported 
part  time  as  Graduate  Research  Assistants  during  the  academic  year;  all  are 
Ph.D.  candidates. 

Developments  during  the  reporting  period 
a)  Published  papers 

Bechhofer,  R.E.:  "Ranking  and  selection  procedures,"  Proceedings  of  the 
Twentieth  Conference  on  the  Design  of  Experiments  in  Army  Research,  Development, 
and  Testing  held  at  Fort  Belvoir,  Virginia,  October  23-25,  1974,  AR0  Report  75-2, 
pp.  929-949. 

Blumenthal,  S.:  "Sequential  estimation  of  the  largest  normal  mean  when  the 
variance  is  unknown,"  Communications  in  Statistics,  Vol.  4,  No.  7,  July  1975, 
pp.  655-669. 


b)  Papers  accepted  for  publication 

Bechhofer,  R.E.,  Santner,  T.J.,  and  Turnbull,  B.W.:  "Selecting  the  largest 
interaction  in  a  two- factor  experiment."  To  appear  in  the  Proceedings  of  the 
Second  Symposium  on  Statistical  Decision  Theory  and  Related  Topics  held  at 
Purdue  University,  May  17-19,  1976. 

Turnbull,  3 . W. :  "The  empirical  distribution  function  with  arbitrarily  grouped, 
censored,  and  truncated  data."  To  appear  in  the  Journal  of  the  Royal  Statistical 
Society ,  3_,  Vol.  38,  No.  3. 

Turnbull,  B "Multiple  decision  rules  for  comparing  several  populations 
with  a  fixed  known  standard."  To  appear  in  Communications  in  Statistics. 


c)  Papers  submitted  for  publication  and  currently  being  revised  at  the  request 

of  the  editor 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (ktl)-decision  single-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case 
of  common  known  variance,"  and  "A  (ktl)-decision  two-stage  selection  procedure 
for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case  of  common 
unknown  variance."  These  papers  have  been  combined  at  the  request  of  the  editor, 
and  the  revised  paper  has  been  resubmitted. 

d)  Papers  submitted  for  publication 

Tamhane,  A.C.:  "On  2-  and  3-stage  screening  procedures  for  selecting  the 
population  having  the  largest  mean  from  k  normal  populations  with  a  common 
unknown  variance." 

Turnbull,  B.W.  and  Weiss,  L. :  "A  likelihood  ratio  statistic  for  testing 
goodness  of  fit  with  randomly  censored  data." 


e)  Technical  reports 

Awate,  P. :  "Dynamic  programming  with  negative  rewards  and  average  reward 
criterion,"  TR  251,  May  1975. 

3echhofer,  R.E.  and  Turnbull,  B.W.:  "A  (k+1  )-decision  two-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case 
of  common  unknown  variance,"  TR  256,  May  1975. 

Turnbull,  B.W. :  "Multiple  decision  rules  for  comparing  several  populations 
with  a  fixed  known  standard,"  TR  257,  June  1975. 

Tamhane,  A.C. :  "On  minimax  multistage  elimination  type  rules  for  selecting 
the  largest  normal  mean,"  TR  259,  May  1975. 

Tamhane,  A.C. :  "On  minimax  two-stage  permanent  elimination  type  procedure 
for  selecting  the  smallest  normal  variance,"  TR  260,  June  1975. 

Gelber,  R.D. :  "A  sequential  goodness-of-fit  test  for  composite  hypotheses 
involving  unknown  scale  and  location  parameters,"  TR  266,  August  1975. 

Bechhofer,  R.E.,  Santner,  T. J. ,  and  Turnbull,  B.W. :  "An  application  of 
majorization  to  the  problem  of  selecting  the  largest  interaction  in  a  two-factor 
experiment,"  TR  292,  May  1976. 


"The  empirical  distribution  function  with  arbitrarily  grouped 
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Turnbull,  3.W. : 
censored,  and  truncated  data,"  TR  305,  June  1976. 

Turnbull,  B.W.  and  Weiss,  L. :  "A  likelihood  ratio  statistic  for  testing  good¬ 
ness  of  fit  with  randomly  censored  data,"  TR  307,  August  1976. 

Jakobovits,  R.H. :  "Goodness  of  fit  tests  for  composite  hypotheses  based  on  an 
increasing  number  of  order  statistics,"  TR  310,  September  1976. 

f)  Papers  in  preparation 

Bechhofer,  R.E.  and  Ramberg,  J.S.:  "A  comparison  of  the  performance  character¬ 
istics  of  two  procedures  for  ranking  means  of  normal  populations." 

Bechhofer,  R.E.  and  Tamhane,  A.C. :  "Optimal  allocation  of  observations  when 
comparing  several  treatments  with  a  control  (IV):  Incomplete  block  designs." 

Bechhofer,  R.E.  and  Turnbull,  B.W. :  "Chebyshev  type  lower  bounds  for  the 
probability  of  correct  selection,  I:  the  location  problem  with  one  observation 
from  each  of  two  populations. " 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (k+l)-decision  single-stage  procedure 
for  selecting  the  production  process  with  the  highest  fraction  of  conforming 
product." 

Kaspi,  H.  and  Wong,  T. :  "On  the  performance  characteristics  of  the  Hoel- 
Mazumdar  elimination  procedure  for  Poisson  processes ,  with  applications  to  clinical 
trials . " 

Santner,  T. J. :  "Contributions  to  the  problem  of  selecting  the  largest  inter¬ 
action  in  factorial  experiments." 

J  Turnbull,  B.W.,  Kaspi,  H. ,  and  Smith,  R. :  "Sequential  allocation  in  clinical 
trials  for  choosing  the  best  of  several  normal  populations." 

•>  Tamhane,  A.C.  and  Bechhofer,  R.E.  :  "A  minimax  permanent  elimination  type  pro¬ 
cedure  for  selecting  the  largest  normal  mean  (common  known  variance)." 


Professional  activities  of  Dr.  Bechhofer  during  the  reporting  period 

a)  Director  of  School  of  Operations  Research  &  Industrial  Engineering,  Cornell 
University. 

b)  Member  of  the  Advisory  Board  of  the  Section  on  Physical  and  Engineering 
Sciences,  American  Statistical  Association. 
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c)  Member  of  the  Committee  on  Statistics  in  the  Physical  Sciences  of  the 
Bernoulli  Society  for  Mathematical  Statistics  and  Probability. 

d)  Refereed  research  proposals  for  Army  Research  Office,  Durham,  and  for 
National  Research  Council,  Canada. 

e)  Gave  an  invited  talk  "Ranking  and  selection  procedures"  before  the 
Washington  Statistical  Society  Chapter  of  the  American  Statistical  Association, 
October  3,  1975. 

f)  Panelist  at  Twenty-first  Conference  on  the  Design  of  Experiments  in  Army 
Research,  Development,  and  Testing  held  at  Walter  Reed  Army  Medical  Center, 
Washington,  D.C.,  October  22-24,  1975. 

g)  Member  of  Visiting  Committee  on  Operations  Research,  Tel-Aviv  University, 
Israel.  Visit  took  place  December  8-11,  1975. 

h)  Gave  seminars  on  "Ranking  and  selection  procedures"  before  the  Department 
of  Statistics,  the  Hebrew  University  and  the  Faculty  of  Industrial  and  Management 
Engineering  of  the  Technion,  Israel,  on  December  17  and  18,  1975,  respectively. 
Also,  at  the  Department  of  Statistics,  Ohio  State  University,  January  21,  1976, 
Department  of  Mathematics,  University  of  Toronto,  Canada,  March  4,  1976; 

Southern  Ontario  Chapter  of  the  American  Statisticsl  Association  and  Department 
of  Applied  Mathematics,  McMaster  University,  Canada,  April  22,  1976. 
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FIRST  ANNUAL  REPORT  ON  CONTRACT  N00014-75-C-0586 


"STATISTICAL  ENGINEERING" 


by 


Robert  E.  Bechhofer 
Principal  Investigator 


School  of  Operations  Research  and  Industrial  Engineering 
College  of  Engineering 
Cornell  University 
Ithaca,  New  York  14853 


FIFTH  (AND  FINAL)  ANNUAL  REPORT  ON  CONTRACT  N00014-67-A-0077-0020 


AND  FIRST  ANNUAL  REPORT  ON  CONTRACT  N00014-75-C-0586 

This  is  the  fifth  (and  final)  annual  report  on  Contract  N00014-67-A-0077-0020, 
and  first  annual  report  on  Contract  N00014-75-C-0586 ,  both  contracts  being  titled 
"Statistical  Engineering."  The  first  contract  was  initiated  on  February  1,  1971 
and  (with  its  third  renewal,  and  an  extension  without  cost)  continued  through 
June  30,  1975;  the  second  contract  was  initiated  on  June  1,  1975  and  continues 
through  May  31,  1976.  The  present  report  covers  the  one-year  period  September  1,  1974 
through  August  31,  1975.  (The  reporting  period  for  the  present  report  therefore 
overlaps  the  reporting  period  for  the  fourth  annual  report  on  the  first  contract . ) 
During  summer  1975  the  contracts  provided  partial  support  to  the  Principal  Investi¬ 
gator,  Dr.  Robert  Bechhofer,  to  a  Research  Associate,  Dr.  Bruce  Turnbull  of  Oxford 
University  (who  visited  Cornell  University  during  this  period),  and  to  several 
Graduate  Research  Assistants.  In  addition  Messrs.  Jose  Kreimerman  and  A jit  Tamhane 
were  supported  part  time  during  fall  1974,  and  Messrs.  Richard  Gelber  and 
Ray  Jakobovits  were  supported  part  time  during  spring  1975;  these  four  Graduate 

m 

Research  Assistants  are  all  Ph.D.  candidates. 

Developments  during  the  reporting  period 

a)  Published  papers 

A  list  of  papers  supported  by  the  first  contract  and  the  predecessor  contract) 
and  published  during  the  reporting  period  follows: 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "An  iterated  integral  representation  for  a 
multivariate  normal  integral  having  block  covariance  structure,"  Biometrika, 

Vol.  61,  No.  3,  pp.  615-619,  1974. 

Miller,  D.R. :  "Limit  theorems  for  path-functionals  of  regenerative  processes," 
Stochastic  Processes  and  Their  Applications,  Vol.  2,  pp.  141-161,  North-Holland 
Publishing  Co. ,  1974. 

b)  Paper  accepted  for  publication 

Bechhofer,  R.E.:  "Ranking  and  selection  procedures."  To  appear  in  the 
Proceedings  of  the  Twentieth  Conference  on  the  Design  of  Experiments  in  Array 
Research,  Development,  and  Testing  held  at  Fort  Belvoir,  Virginia,  October  23-25,  1974. 


Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (ktl)-decision  single-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case  of 
common  known  variance." 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (k+l)-decision  two-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case  of 
common  unknown  variance . " 

Frischtak,  R. :  "Selection  of  subclasses  of  variates  based  on  a  measure  of 
association. " 

Tamhane,  A.C.:  "On  2-  and  3-stage  screening  procedures  for  selecting  the 
population  having  the  largest  mean  from  k  normal  populations  with  a  common 
unknown  variance." 

Turnbull,  B.W. :  "Multiple  decision  rules  for  comparing  several  populations 
with  a  fixed  known  standard. " 

d)  Technical  reports 

Bechhofer,  R.E. :  "A  two-sample  procedure  for  selecting  the  population  with 
the  largest  mean  from  several  normal  populations  with  unknown  variances:  some 
comments  on  Ofosu's  paper,"  TR  233,  October  1974. 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (ktl)-decision  single-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  to own  standard:  the  case  of 
common  known  variance,"  TR  242,  December  1974  (revised  May  1975). 

Kreimerman,  J. :  "A  bivariate  test  of  goodness  of  fit  based  on  a  gradually 
increasing  number  of  order  statistics,"  TR  250,  March  1975. 

Awate,  P. :  "Dynamic  programming  with  negative  rewards  and  average  reward 
criterion,"  TR  251,  May  1975. 

Bechhofer,  R.E.  and  Turnbull,  B.W. :  "A  (ktl)-decision  two-stage  selection 
procedure  for  comparing  k  normal  means  with  a  fixed  known  standard:  the  case  of 
common  unknown  variances,"  TR  256,  May  1975. 

Turnbull,  B.W. :  "Multiple  decision  rules  for  comparing  several  populations 
with  a  fixed  known  standard,"  TR  257,  June  1975. 

Tamhane,  A.C. :  "On  minimax  multistage  elimination  type  rules  for  selecting 
the  largest  normal  mean,"  TR  259,  May  1975. 

Tamhane,  A.C.:  "A  minimax  two-stage  permanent  elimination  type  procedure  for 
selecting  the  smallest  normal  variance,"  TR  260,  June  1975. 

Gelber,  R. D. :  "A  sequential  goodness-of-fit  test  for  composite  hypotheses 
involving  unknown  scale  and  location  parameters,"  TR  266,  August  1975. 
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e)  Papers  in  preparation 

Awate,  P. ,  Bechhofer,  R.E.,  and  Tamhane,  A.C.:  ”A  maximin  procedure  for 
ranking  means  of  normal  populations  with  known  unequal  variances." 

Bechhofer,  R.E.:  "On  designing  multi-factor  experiments  to  identify  the 
treatment  combination  associated  with  the  largest  interaction." 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "Chebyshev  type  lower  bounds  for  the 
probability  of  correct  selection,  I:  the  location  problem  with  one  observation 
from  each  of  two  populations." 

Bechhofer,  R.E.  and  Turnbull,  B.W.:  "A  (k+l)-decision  single-stage  proce¬ 
dure  for  selecting  the  production  process  with  the  highest  fraction  of  conforming 
product . " 

Bechhofer,  R.E.  and  Turnbull,  B.W. :  "A  (k+l)-decision  single-stage  procedure 
for  selecting  the  normal  population  with  mean  closest  to  a  fixed  known  standard." 

Tamhane,  A.C.  and  Bechhofer,  R.E.:  "Optimal  design  for  comparing  several 
treatments  with  a  control  using  incomplete  blocks." 

Tamhane,  A.C.  and  Bechhofer,  R.E.:  "A  minimax  2-stage  permanent  elimination 
type  procedure  for  selecting  the  largest  normal  mean  (common  known  variance)." 

Professional  activities  of  Dr.  Bechhofer  during  the  reporting  period 

a)  Member  of  the  Committee  on  Summer  Research  Institutes  of  the  Institute  of 
Mathematical  Statistics. 

b)  Member  of  the  Advisory  Board  of  the  Section  on  Physical  and  Engineering 
Sciences,  American  Statistical  Association. 

c)  Reviewed  research  proposals  for  the  National  Science  Foundation  and  the 
Army  Research  Office,  Durham. 

d)  Presented  an  invited  paper,  "Ranking  and  Selection  Procedures"  at  the 
Twentieth  Conference  on  the  Design  of  Experiments  in  Army  Research,  Development, 
and  Testing  held  at  Fort  Belvoir,  Virginia,  October  23-25,  1974. 

e)  Invited  to  give  a  paper  at  the  Symposium  on  Statistical  Decision  Theory 
and  Related  Topics  to  be  held  at  Purdue  University  on  May  17-19,  1976. 

f)  Elected  an  Ordinary  Member  of  the  International  Statistical  Institute  — 
one  of  ten  so  elected  from  the  United  States. 

g)  Named  Director  of  a  newly-reorganized  School  of  Operations  Research  and 
Industrial  Engineering,  Cornell  University. 
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